Shake2Talk: Multimodal Messaging for Interpersonal Communication

This paper explores the possibilities of using audio and haptics for interpersonal communication via mobile devices. Drawing on the literature on current messaging practises, a new concept for multimodal messaging has been designed and developed. The Shake2Talk system allows users to construct audio-tactile messages through simple gesture interactions, and send these messages to other people. Such messages could be used to communicate a range of meanings, from the practical (e.g. “home safely”, represented by the sound and sensation of a key turning in a lock) to the emotional (e.g. “thinking of you” represented by a heartbeat). This paper presents the background to this work, the system design and implementation and a plan for evaluation.